Repository navigation
Conversation
|
I suggest you rebase and see if CI will become green. There are compiler errors that might have been fixed. |
a90349e to
c24476b
Compare
|
Rebased onto current main, it applied cleanly and donate_cpu_lib_test.py still passes locally. Let's see what CI says now. Separately, could one of you create a Trac ticket for this? I don't have access, and I'll put the number in the title once it exists. |
dmcppcheck
left a comment
There was a problem hiding this comment.
AI review
The fix is incomplete when Python has no tarfile.data_filter, which is the case it is meant for. The new realpath check runs on the member from the loop, but extraction still calls tf.extract(member.name, ...). Extracting by name looks up the last member with that name in the archive, so a harmless first entry can pass the check while a later entry with the same name, such as a hardlink or symlink that points outside temp, gets extracted. Hardlinks also defeat the realpath check of later members: realpath cannot tell that a file inside temp is a hardlink to a file outside it, and a later regular-file member that writes through that path writes to the outside file. On Python versions with data_filter (3.12 here), the filter still catches this. The fix is to pass the member object to tf.extract. Apart from that I found no new problems.
Comments
| # | Comment | By | Where | Status |
|---|---|---|---|---|
| 1 | extract(member.name) extracts the last member with that name, not the one that was checked | AI review | tools/donate_cpu_lib.py:409 |
🆕 New |
@Nussu06, please fix, resolve or reply to the open and new comments: #1.
Testing
- I did not build
cppcheckortestrunner. The PR only changestools/donate_cpu_lib.pyand its Python test, so no C++ code is affected. PYTHONPATH=tools python3 -m pytest -Werror -q test/tools/donate_cpu_lib_test.pyinpr/with Python 3.12.3: 3 passed.- The same test file run against
base/:test_unpack_package_outside_tempfails as expected, and the other 2 pass. - I checked how name lookup behaves with duplicate member names, which is what the finding depends on:
# scratch/dup.py
import io, tarfile, tempfile, os
d = tempfile.mkdtemp(); p = os.path.join(d, 'x.tar')
with tarfile.open(p, 'w') as tf:
for c in (b'first\n', b'second\n'):
i = tarfile.TarInfo('a.c'); i.size = len(c); tf.addfile(i, io.BytesIO(c))
with tarfile.open(p) as tf:
for m in tf:
tf.extract(m.name, d, filter='data')
print(open(os.path.join(d, 'a.c')).read().strip())
breakpython3 -I dup.py prints second. On the first loop iteration, tf.extract(m.name) extracted the second member, not the one the loop was on.
- I did not reproduce a full escape from
temp. The impact on Python versions withoutdata_filtercomes from reading the code: the check runs on one member while another member with the same name is extracted.
Automated review of c24476b by Claude. The pull request was built and tested locally.
| continue | ||
| try: | ||
| tf.extract(member.name, temp_path) | ||
| tf.extract(member.name, temp_path, **extract_args) |
There was a problem hiding this comment.
AI review
The safety check above validates member, but tf.extract(member.name, ...) goes through getmember(name), which returns the last member with that name in the archive. A tarball can contain a harmless pkg/a.c first and a later pkg/a.c that is a hardlink or symlink pointing outside temp. The first entry passes the check, and extracting it creates the later, unchecked link. The later entry is then skipped, but by that point the link has already been created.
With filter='data' the filter still rejects the link, but on Python versions without tarfile.data_filter (the case this PR protects) nothing stops it. A hardlink is also invisible to the realpath check, so a later regular-file member that writes through that path (for example through a directory symlink that stays inside temp) writes into the file outside temp.
Passing the member object fixes this:
tf.extract(member, temp_path, **extract_args)It would be worth adding a duplicate-name case to test_unpack_package_outside_temp.
There was a problem hiding this comment.
Good catch, extracting by name picks up the last entry with that name, so the check and the extraction could be looking at different members. It now passes the member object to tf.extract. I added test_unpack_package_duplicate_name, which has a regular pkg/sym.c followed by a same-named symlink to a file outside temp, and the same for a hardlink. It fails with the old code on Python 3.10.11 (no data_filter) and passes with the fix on 3.10 and 3.11. The Trac ticket is still open on my side, if someone can create one I'll put the number in the title.
dmcppcheck
left a comment
There was a problem hiding this comment.
AI review
On Python versions without tarfile.data_filter, which this PR is meant to protect, a symlink that points outside temp can still be created. The symlink target is resolved against os.path.dirname(member_path), but member_path has already gone through realpath. If the archive first creates pkg/inc.h as a symlink to a deep path inside temp, a second pkg/inc.h symlink to ../../outside is checked against that deep directory and looks inside temp. tarfile then replaces the link at temp/pkg/inc.h, so the new link resolves relative to temp/pkg and points outside. Later members that write through the link are still skipped. But cppcheck follows directory symlinks (stat in cli/filelister.cpp), so donate-cpu would analyse files outside the package, and their results would be uploaded. I found no other new problems, and the earlier comment about extracting by name is fixed.
Comments
| # | Comment | By | Where | Status |
|---|---|---|---|---|
| 1 | extract(member.name) extracts the last member with that name, not the checked one | AI review | general | ✅ Fixed |
| 2 | Symlink target is resolved from the realpath of an existing link, so a replacing symlink can point outside temp | AI review | tools/donate_cpu_lib.py:386 |
🆕 New |
@Nussu06, please fix, resolve or reply to the open and new comments: #2.
Testing
- I did not build
cppcheckortestrunner, because the PR only changestools/donate_cpu_lib.pyand its Python test. PYTHONPATH=tools python3 -m pytest -q test/tools/donate_cpu_lib_test.pyinpr/with Python 3.12.3: 4 passed. They also pass withtarfile.data_filterdeleted, to simulate an older Python.- I reproduced the symlink escape with Python 3.12 and
data_filterremoved:
# scratch/esc.py — run: python3 -I esc.py /work/pr/tools nofilter
import io, os, sys, tarfile, tempfile
sys.path.insert(0, sys.argv[1])
if len(sys.argv) > 2:
del tarfile.data_filter # simulate Python without extraction filters
import donate_cpu_lib as d
w = tempfile.mkdtemp(); out = os.path.join(w, 'outside'); os.mkdir(out)
open(os.path.join(out, 'secret.c'), 'w').write('int secret;\n')
tgz = os.path.join(w, 'p.tgz')
def add(tf, name, link=None):
i = tarfile.TarInfo(name); data = b'int x;\n'
if link: i.type = tarfile.SYMTYPE; i.linkname = link; data = b''
else: i.size = len(data)
tf.addfile(i, io.BytesIO(data))
with tarfile.open(tgz, 'w:gz') as tf:
add(tf, 'pkg/main.c')
add(tf, 'pkg/inc.h', 'a/b/c') # inside temp
add(tf, 'pkg/inc.h', '../../outside') # checked relative to temp/pkg/a/b, created in temp/pkg
tp, found = d.unpack_package(w, tgz)
l = os.path.join(tp, 'pkg', 'inc.h')
print('link:', os.readlink(l) if os.path.islink(l) else None, '->', os.path.realpath(l))
print('listing via temp:', os.listdir(l) if os.path.isdir(l) else None)Output without the filter:
extracted 3 of 3 files (skipped 0)
link: ../../outside -> /tmp/tmpbyf2t84d/outside
listing via temp: ['secret.c']
With filter='data' (the normal 3.12 run), the second link is rejected and temp/pkg/inc.h stays a/b/c.
Automated review of 998d32a by Claude. The pull request was built and tested locally.
| member_path = os.path.realpath(os.path.join(temp_path, member.name)) | ||
| link_path = None | ||
| if member.issym(): | ||
| link_path = os.path.realpath(os.path.join(os.path.dirname(member_path), member.linkname)) |
There was a problem hiding this comment.
AI review
member_path has already gone through realpath. If an earlier member created a symlink with the same name (for example pkg/inc.h -> a/b/c), os.path.dirname(member_path) is temp/pkg/a/b, not temp/pkg. tarfile's makelink unlinks the old link and creates the new one at temp/pkg/inc.h, so its relative target resolves from temp/pkg.
Example: a second member pkg/inc.h -> ../../outside passes this check, because from the deep directory it resolves to temp/outside. On disk it points to work_path/outside. I reproduced this with data_filter removed: all 3 members were extracted, and os.listdir(temp/pkg/inc.h) listed the files in the outside folder. cppcheck follows directory symlinks when it scans temp (stat in cli/filelister.cpp), so files outside the package would be analysed and reported.
Resolve the link target from the real parent directory of the unresolved destination instead:
parent = os.path.realpath(os.path.dirname(os.path.join(temp_path, member.name)))
link_path = os.path.realpath(os.path.join(parent, member.linkname))This case could be added to test_unpack_package_duplicate_name.
unpack_package in tools/donate_cpu_lib.py only skips archive members whose name starts with '/' or '..', so a package tarball containing pkg/../../outside/x.c, or a symlink pkg/inc.h pointing at another directory followed by pkg/inc.h/x.c, writes source files outside work_path/temp when donate-cpu.py, test-my-pr.py or compare-valueflow-options.py unpack it on Python before 3.14 (3.14 refuses these by default, but the uncaught OutsideDestinationError aborts the run). Each member's destination, and the target of symlink and hardlink members, is now resolved with realpath and skipped unless it stays inside the temp folder, and filter='data' is passed where tarfile supports it; links that stay inside the package, such as include/a.h -> ../src/a.h, are still extracted. Added test_unpack_package_outside_temp, which fails on main with Python 3.9 and 3.12 and passes with the fix on 3.9, 3.12 and 3.14, and bumped CLIENT_VERSION to 1.4.1. I don't have Trac access, so could you create a ticket for this? I'll update the title once there is a number.